Detection system, detection method, and computer program

ABSTRACT

A detection system (10) includes: an acquisition unit (110) configured to acquire an image including a living body; and a detection unit (120) configured to detect, from the image, a feature figure corresponding to an appropriately circular first part on the living body, and feature points corresponding to a second part around the first part on the living body. According to such a detection system, the first part and the second part with different features in shape can be individually detected appropriately.

This application is a National Stage Entry of PCT/JP2020/034900 filed onSep. 15, 2020, the contents of all of which are incorporated herein byreference, in their entirety.

TECHNICAL FIELD

The present disclosure relates to a technical field of a detectionsystem, a detection method, and a computer program that detect part of aliving body from an image.

BACKGROUND ART

As a system of this type, there is known a system that detects an areaaround an eye of a living body from an image. For example, PatentReference 1 discloses that a pupil circle and an iris circle aredetected from an image. Patent Reference 2 discloses that a face isdetected from an image, and an eye is detected based on positioninformation on the face. Patent Reference 3 discloses that featurepoints are extracted from a face image. Patent Reference 4 disclosesthat circular areas are detected in a ROI (Region of Interest), and aplurality of areas to be candidates for the iris are detected.

CITATION LIST Patent Literature Patent Literature 1

-   Japanese Patent Laid-Open No. 2003-317102 A

Patent Literature 2

-   Japanese Patent Laid-Open No. 2007-213377 A

Patent Literature 3

-   Japanese Patent Laid-Open No. 2014-075098 A

Patent Literature 4

-   Japanese Patent Laid-Open No. 2018-045437 A

SUMMARY Technical Problem

In detection where an image is an input, a conceivable method is ofdetecting feature points and a feature figure. However, none of theprior art references cited above mention detecting both feature pointsand a feature figure, and there is room for improvement.

An object of the present disclosure is to provide a detection system, adetection method, and a computer program that can solve the aboveproblem.

Solution to Problem

A detection system according to an example aspect of the presentinvention includes: an acquisition unit configured to acquire an imageincluding a living body; and a detection unit configured to detect, fromthe image, a feature figure corresponding to an appropriately circularfirst part on the living body, and feature points corresponding to asecond part around the first part on the living body.

A detection method according to an example aspect of the presentinvention includes: acquiring an image including a living body; anddetecting, from the image, a feature figure corresponding to anappropriately circular first part on the living body, and feature pointscorresponding to a second part around the first part on the living body.

A computer program according to an example aspect of the presentinvention allows a computer to: acquire an image including a livingbody; and detect, from the image, a feature figure corresponding to anappropriately circular first part on the living body, and feature pointscorresponding to a second part around the first part on the living body.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing a hardware configuration of adetection system according to a first embodiment.

FIG. 2 is a block diagram showing a functional configuration of thedetection system according to the first embodiment.

FIG. 3 is a flowchart showing a flow of operation of the detectionsystem according to the first embodiment.

FIG. 4 shows examples of a feature figure detected by the detectionsystem according to the first embodiment.

FIG. 5 is a block diagram showing a functional configuration of adetection system according to a second embodiment.

FIG. 6 is a flowchart showing a flow of operation of the detectionsystem according to the second embodiment.

FIG. 7 shows an example of detection of feature figures and featurepoints by the detection system according to the second embodiment.

FIG. 8 is a block diagram showing a functional configuration of adetection system according to a third embodiment.

FIG. 9 is a flowchart showing a flow of operation of the detectionsystem according to the third embodiment.

FIG. 10 shows an example of a method for estimating a line of sight by adetection system according to a fourth embodiment.

FIG. 11 is a block diagram showing a functional configuration of adetection system according to a fifth embodiment.

FIG. 12 is a flowchart showing a flow of operation of the detectionsystem according to the fifth embodiment.

FIG. 13 shows a specific example of operation performed by the detectionsystem according to the fifth embodiment.

FIG. 14 is a block diagram showing a functional configuration of adetection system according to a sixth embodiment.

FIG. 15 is a flowchart showing a flow of operation of the detectionsystem according to the sixth embodiment.

FIG. 16 is a diagram (version 1) showing an example of display offeature points and feature figures on a display unit.

FIG. 17 is a diagram (version 2) showing an example of display of thefeature points and the feature figures on the display unit.

FIG. 18 is a diagram (version 3) showing an example of display of thefeature points and the feature figures on the display unit.

FIG. 19 is a diagram (version 4) showing an example of display of thefeature points and the feature figures on the display unit.

FIG. 20 is a diagram (version 5) showing an example of display of thefeature points and the feature figures on the display unit.

FIG. 21 is a block diagram showing a functional configuration of adetection system according to a seventh embodiment.

FIG. 22 is a flowchart showing a flow of operation of the detectionsystem according to the seventh embodiment.

DESCRIPTION OF EXAMPLE EMBODIMENTS

Hereinafter, embodiments of a detection system, a detection method, anda computer program will be described with reference to drawings.

First Embodiment

A detection system according to a first embodiment is described withreference to FIGS. 1 to 4 .

(Hardware Configuration)

First, a hardware configuration of the detection system according to thefirst embodiment is described with reference to FIG. 1 . FIG. 1 is ablock diagram showing the hardware configuration of the detection systemaccording to the first embodiment.

As shown in FIG. 1 , the detection system 10 according to the firstembodiment includes a processor 11, a RAM (Random Access Memory) 12, aROM (Read Only Memory) 13, and a storage device 14. The detection system10 may further include an input device 15 and an output device 16. Theprocessor 11, the RAM 12, the ROM 13, the storage device 14, the inputdevice 15, and the output device 16 are connected to each other througha data bus 17.

The processor 11 reads a computer program. For example, the processor 11is configured to read the computer program stored in at least one of theRAM 12, the ROM 13, and the storage device 14. Alternatively, theprocessor 11 may read the computer program stored in a computer-readablerecording medium, by using an undepicted recording media reader. Theprocessor 11 may acquire (that is, may read) the computer program, via anetwork interface, from an undepicted device disposed outside of thedetection system 10. The processor 11 controls the RAM 12, the storagedevice 14, the input device 15, and the output device 16 by executingthe read computer program. In the present embodiment in particular, whenthe processor 11 executes the read computer program, a functional blockfor detecting part of a living body from an image is implemented in theprocessor 11. For the processor 11, one of a CPU (Central ProcessingUnit), a GPU (Graphics Processing Unit), an FPGA (field-programmablegate array), a DSP (Demand-Side Platform), and an ASIC (ApplicationSpecific Integrated Circuit) may be used, or two or more thereof may beused in parallel.

The RAM 12 transitorily stores the computer program to be executed bythe processor 11. The RAM 12 transitorily stores data transitorily usedby the processor 11 when the processor 11 executes the computer program.The RAM 12 may be, for example, a D-RAM (Dynamic RAM).

The ROM 13 stores the computer program to be executed by the processor11. The ROM 13 may store other fixed data. The ROM 13 may be, forexample, a P-ROM (Programmable ROM).

The storage device 14 stores data that the detection system 10 retainsfor a long time. The storage device 14 may operate as a transitorystorage device for the processor 11. The storage device 14 may includeat least one of, for example, a hard disk device, a magneto-optical diskdevice, an SSD (Solid State Drive), and a disk array device.

The input device 15 is a device that receives an input instruction froma user of the detection system 10. The input device 15 may include atleast one of, for example, a keyboard, a mouse, and a touch panel.

The output device 16 is a device that outputs information related to thedetection system 10 to the outside. For example, the output device 16may be a display device (for example, a display) capable of displayingthe information related to the detection system 10.

(Functional Configuration)

Next, a functional configuration of the detection system 10 according tothe first embodiment is described with reference to FIG. 2 . FIG. 2 is ablock diagram showing the functional configuration of the detectionsystem according to the first embodiment.

In FIG. 2 , the detection system 10 according to the first embodiment isconfigured as a system that detects part of a living body from an image.The detection system 10 includes, as processing blocks for implementingfunctions of the detection system 10, or as physical processingcircuitry, an image acquisition unit 110 and a detection unit 120. Theimage acquisition unit 110 and the detection unit 120 can be implementedby, for example, the above-described processor 11 (see FIG. 1 ).

The image acquisition unit 110 is configured to be able to acquire animage inputted into the detection system 10 (that is, an image subjectto detection). The image acquisition unit 110 may include anaccumulation unit configured to accumulate acquired images. Aconfiguration is made such that information related to the imageacquired by the image acquisition unit 110 is outputted to the detectionunit 120.

The detection unit 120 is configured to be able to detect part of aliving body from the image acquired by the image acquisition unit 110.Specifically, the detection unit 120 is configured to be able to detecta feature figure corresponding to a first part of the living body, andfeature points corresponding to a second part of the living body. The“first part” here is a part having an approximately circular shape onthe living body. On the other hand, the “second part” is a part locatedaround the first part on the living body. Note that it may be presetwhich parts of a living body are the first part and the second part,respectively. In such a case, a plurality of parts of different typesmay be set as first parts, and a plurality of parts of different typesmay be set as second parts. The detection unit 120 may include afunction of outputting information related to the detected featurepoints and feature figure.

(Flow of Operation)

Next, a flow of operation of the detection system 10 according to thefirst embodiment is described with reference to FIG. 3 . FIG. 3 is aflowchart showing the flow of the operation of the detection systemaccording to the first embodiment.

As shown in FIG. 3 , when the detection system 10 according to the firstembodiment operates, first, the image acquisition unit 110 acquires animage (step S101).

Subsequently, the detection unit 120 detects a feature figurecorresponding to the first part from the image acquired by the imageacquisition unit 110 (step S102). The detection unit 120 further detectsfeature points corresponding to the second part from the image acquiredby the image acquisition unit 110 (step S103). The detected featurefigure and feature points can be represented by coordinates, amathematical formula, or the like. Note that the processes in steps S102and S103 may be sequentially executed, or may be simultaneously executedin parallel. In other words, order in which the feature figurecorresponding to the first part and the feature points corresponding tothe second part are detected is not limited, and a configuration may bemade such that the feature figure and the feature points are detected atthe same time.

(Specific Examples of Feature Figure)

Next, specific examples of the feature figure detected by the detectionsystem 10 according to the first embodiment are described with referenceto FIG. 4 . FIG. 4 shows examples of the feature figure detected by thedetection system according to the first embodiment.

As shown in FIG. 4 , in the detection system 10 according to the firstembodiment, the detection unit 120 detects a circle (including anellipse), as the feature figure corresponding to the first part. Circlesdetected by the detection unit 120 may include, in addition to a regularcircle, a vertically long ellipse, a horizontally long ellipse, and anoblique ellipse (that is, an ellipse rotated at an arbitrary angle). Itmay be preset what kinds of circles are to be detected in actuality. Forexample, a shape corresponding to a part of a living body intended to bedetected may be set. Moreover, the detection unit 120 may be configuredto be able to detect a partially broken circle or a circle partiallyobstructed from view.

Technical Effects

Next, technical effects achieved by the detection system 10 according tothe first embodiment are described.

As described with FIGS. 1 to 3 , in the detection system 10 according tothe first embodiment, a feature figure corresponding to the first partand feature points corresponding to the second part are detected from animage. In other words, the first part and the second part are detectedthrough different methods. Thus, the first part and the second part,which have different features in shape, can be individually detectedappropriately from an image obtained by picking up an image of a livingbody.

Second Embodiment

A detection system 10 according to a second embodiment is described withreference to FIGS. 5 to 7 . Note that the second embodiment is differentfrom the above-described first embodiment only in part of configurationand operation, and, for example, a hardware configuration may be similarto that of the first embodiment (see FIG. 1 ). Accordingly, in thefollowing, a description of part overlapping with the embodimentdescribed already is omitted as appropriate.

(Functional Configuration)

First, a functional configuration of the detection system 10 accordingto the second embodiment is described with reference to FIG. 5 . FIG. 5is a block diagram showing the functional configuration of the detectionsystem according to the second embodiment. Note that in FIG. 5 ,elements similar to the constitutional elements shown in FIG. 2 aredenoted by the same reference numbers as in FIG. 2 .

As shown in FIG. 5 , the detection system 10 according to the secondembodiment includes, as processing blocks for implementing functions ofthe detection system 10, or as physical processing circuitry, an imageacquisition unit 110, a detection unit 120, and an iris recognition unit130. In other words, the detection system 10 according to the secondembodiment includes the iris recognition unit 130, in addition to theconstitutional elements in the first embodiment (see FIG. 2 ). Note thatthe iris recognition unit 130 can be implemented by, for example, theabove-described processor 11 (see FIG. 1 ).

The iris recognition unit 130 is configured to be able to execute irisrecognition by using feature points and a feature figure detected by thedetection unit 120. For example, the iris recognition unit 130 isconfigured to be able to identify an iris region, based on the featurepoints corresponding to eyelids, which are an example of the secondpart, and on the feature figures corresponding to an iris and a pupil,which are examples of the first parts (see FIG. 7 ), and able to executean iris recognition process using the iris region. The iris recognitionunit 130 may include a function of outputting a result of the irisrecognition. Moreover, the iris recognition unit 130 may be configuredto have part of the iris recognition process be executed outside of thesystem (for example, be executed by an external server, cloud computing,or the like).

(Flow of Operation)

Next a flow of operation of the detection system 10 according to thesecond embodiment is described with reference to FIG. 6 . FIG. 6 is aflowchart showing the flow of the operation of the detection systemaccording to the second embodiment. Note that in FIG. 6 , processessimilar to the processes shown in FIG. 3 are denoted by the samereference numbers as in FIG. 3 .

As shown in FIG. 6 , when the detection system 10 according to thesecond embodiment operates, first, the image acquisition unit 110acquires an image (step S101). Thereafter, the detection unit 120detects feature figures corresponding to the first parts from the imageacquired by the image acquisition unit 110 (step S102). The detectionunit 120 further detects feature points corresponding to the second partfrom the image acquired by the image acquisition unit 110 (step S103).

Subsequently, the iris recognition unit 130 identifies an eyelid region(that is, a region where eyelids exit) from the feature pointscorresponding to the eyelids, and generates a mask over the eyelidregion (step S201). The mask over the eyelid region is used to removethe eyelid region that is not required for the iris recognition (inother words, does not have iris information). Thereafter, the irisrecognition unit 130 identifies an iris region (that is, a region wherethe iris information can be obtained) from the feature figurescorresponding to an iris and a pupil, and executes the iris recognitionusing the iris region (step S202). A detailed description of specificprocessing content of the iris recognition is omitted here because anexisting technique can be adopted as appropriate.

(Example of Detection of Area around Eye)

Next, detection of the feature figures and the feature points by thedetection system 10 according to the second embodiment is described withreference to FIG. 7 . FIG. 7 shows an example of the detection of thefeature figures and the feature points by the detection system accordingto the second embodiment.

As shown in FIG. 7 , in the detection system 10 according to the secondembodiment, the “iris and pupil” are detected as the first parts, andthe “eyelids” are detected as the second part.

The detection unit 120 detects a circle corresponding to the iris and acircle corresponding to the pupil. Note that the detection unit 120 maybe configured to detect only any one of the circle corresponding to theiris and the circle corresponding to the pupil. The iris and the pupilare suitable to be detected as approximately circular feature figuresbecause shapes of the iris and the pupil are nearly circles. If anattempt is made to detect the iris and the pupil as feature points (forexample, as points on a circumference), the number and positions of thepoints depend on a system design and directly affect detection accuracy.However, when the iris and the pupil are detected as circles, positionsof the iris and the pupil can be determined as formulas of the circles.A circle formula is uniquely determined, and therefore does not dependon a system design or affect detection accuracy. In such respects aswell, it can be said that the iris is suitable to be detected as acircle.

Moreover, the detection unit 120 detects a plurality of feature pointsindicating a position (an outline) of the eyelids. In the example shownin the drawing, the detection unit 120 detects two feature pointscorresponding to inner and outer corners of an eye, three feature pointscorresponding to the upper eyelid, and three feature pointscorresponding to the lower eyelid. Note that the above-described numberof the feature points is only an example, and a configuration may bemade such that fewer feature points are detected, or more feature pointsare detected. Eyelids have relatively large individual differences amongliving bodies, and there are considerable differences in shape amongindividuals, such as single eyelids and double eyelids, and upturnedeyes and downturned eyes. Accordingly, the eyelids are suitable to bedetected not as a feature figure but as feature points. Note thatalthough eyelids have individual differences in shape, there is acommonality, which is that eyelids are located around an iris and apupil. Accordingly, if the eyelids are detected along with the featurefigures, the eyelids can be relatively easily detected as featurepoints.

Technical Effects

Next, technical effects achieved by the detection system 10 according tothe second embodiment are described.

As described with FIGS. 5 to 7 , in the detection system 10 according tothe second embodiment, the iris recognition is executed by usingdetected feature figures and feature points. In the present embodimentin particular, since a plurality of parts existing around an eye areappropriately detected, the iris recognition can be executedappropriately. Note that in the example shown in FIG. 5 , sincecoordinates of the inner and outer corners of the eye, and radii of thecircles of the iris and the pupil are known, a ratio of a differencebetween a distance from the inner corner to the outer corner of the eyeand the radius of one of the two circles, to a difference between thedistance and the radius of the other circle may be matched against datain the iris recognition and weighted.

Third Embodiment

A detection system 10 according to a third embodiment is described withreference to FIGS. 8 and 9 . Note that the third embodiment is differentfrom each of the above-described embodiments only in part ofconfiguration and operation, and, for example, a hardware configurationmay be similar to that of the first embodiment (see FIG. 1 ).Accordingly, in the following, a description of part overlapping withthe embodiments described already is omitted as appropriate.

(Functional Configuration)

First, a functional configuration of the detection system 10 accordingto the third embodiment is described with reference to FIG. 8 . FIG. 8is a block diagram showing the functional configuration of the detectionsystem according to the third embodiment. Note that in FIG. 8 , elementssimilar to the constitutional elements shown in FIGS. 2 and 5 aredenoted by the same reference numbers as in FIGS. 2 and 5 .

As shown in FIG. 8 , the detection system 10 according to the thirdembodiment includes, as processing blocks for implementing functions ofthe detection system 10, or as physical processing circuitry, an imageacquisition unit 110, a detection unit 120, and a line-of-sightestimation unit 140. In other words, the detection system 10 accordingto the third embodiment further includes the line-of-sight estimationunit 140, in addition to the constitutional elements in the firstembodiment (see FIG. 2 ). Note that the line-of-sight estimation unit140 can be implemented by, for example, the above-described processor 11(see FIG. 1 ).

The line-of-sight estimation unit 140 is configured to be able toexecute line-of-sight direction estimation by using feature points and afeature figure detected by the detection unit 120. Specifically, theline-of-sight estimation unit 140 is configured to be able to execute aprocess of estimating a direction of a line of sight, based on the“feature points corresponding to the eyelids” and the “feature figurescorresponding to the iris and the pupil” described in the secondembodiment (see FIG. 7 ). Note that the feature points corresponding tothe eyelids and the feature figures corresponding to the iris and thepupil may be detected as described in the second embodiment. Theline-of-sight estimation unit 140 may include a function of outputting aresult of the line-of-sight estimation. Moreover, the line-of-sightestimation unit 140 may be configured to have part of the line-of-sightestimation process be executed outside of the system (for example, beexecuted by an external server, cloud computing, or the like).

(Flow of Operation)

Next, a flow of operation of the detection system 10 according to thethird embodiment is described with reference to FIG. 9 . FIG. 9 is aflowchart showing the flow of the operation of the detection systemaccording to the third embodiment. Note that in FIG. 9 , processessimilar to the processes shown in FIGS. 3 and 6 are denoted by the samereference numbers as in FIGS. 3 and 6 .

As shown in FIG. 9 , when the detection system 10 according to the thirdembodiment operates, first, the image acquisition unit 110 acquires animage (step S101). Thereafter, the detection unit 120 detects featurefigures corresponding to the first parts from the image acquired by theimage acquisition unit 110 (step S102). The detection unit 120 furtherdetects feature points corresponding to the second part from the imageacquired by the image acquisition unit 110 (step S103).

Subsequently, the line-of-sight estimation unit 140 estimates adirection of a line of sight, based on the feature figures and thefeature points (step S301). Note that specific processing content of theline-of-sight direction estimation is described in detail in a fourthembodiment, which will be described later.

Technical Effects

Next, technical effects achieved by the detection system 10 according tothe third embodiment are described.

As described with FIGS. 8 and 9 , in the detection system 10 accordingto the third embodiment, the line-of-sight estimation is executed byusing detected feature figures and feature points. In the presentembodiment in particular, since a part used to estimate the direction ofthe line of sight (for example, each part in and around an eye) isappropriately detected, the direction of the line of sight can beappropriately estimated.

Fourth Embodiment

A detection system 10 according to the fourth embodiment is describedwith reference to FIG. 10 . Note that the fourth embodiment is toillustrate a more specific configuration of the above-described thirdembodiment (that is, a specific method for estimating the direction ofthe line of sight), and a configuration and a flow of operation may besimilar to those of the third embodiment (see FIGS. 8 and 9 ).Accordingly, in the following, a description of part overlapping withthe part described already is omitted as appropriate.

(Calculation of Relative Position)

A method for estimating the direction of the line of sight by thedetection system 10 according to the fourth embodiment is described withreference to FIG. 10 . FIG. 10 shows an example of the method forestimating the line of sight by the detection system according to thefourth embodiment.

As shown in FIG. 10 , in the detection system 10 according to the fourthembodiment, the direction of the line of sight is estimated by usingfeature points corresponding to eyelids and feature figurescorresponding to an iris and a pupil. Note that the feature pointscorresponding to the eyelids and the feature figures corresponding tothe iris and the pupil may be detected as described in the secondembodiment (see FIG. 7 ).

Feature points 1 and 2 in the drawing are feature points correspondingto inner and outer corners of an eye, respectively. Feature points 3 and4 are points at which a median line extended from a line between thefeature points 1 and 2 intersects with the eyelids. Accordingly, thefeature points 1 to 4 keep the same positions unless a direction of aface is changed, even if the eye is turned in any direction. Circles 5and 6, which are the feature figures corresponding to the iris and thepupil, move when a direction of the eye is changed. Accordingly, if arelative relation between positions of the eyelids, which can beidentified from each feature point, and positions of the iris and thepupil, which can be identified from the respective feature figures, isused, the direction of the eye (that is, the direction of the line ofsight) can be estimated.

To estimate the line of sight, a correlation between the relativerelation between the positions of the eyelids and the position of theeye, and which location is currently looked at may be obtainedbeforehand. Such correlations may be calculated as a function, or may becreated as a table.

In calculation of the direction of the line of sight, first, an image isnormalized by using the circle 6 corresponding to the iris. Next, anintersection of a line joining the feature points 1 and 2 with a linejoining the feature points 3 and 4 is set as an origin, and the relativepositional relation between the eyelids and the eye is calculated frominformation on how many pixels the eyelids are far from the origin in x,y directions. Then, the direction of the line of sight is estimated byusing the calculated positional relation between the eyelids and theeye.

Technical Effects

Next, technical effects achieved by the detection system 10 according tothe fourth embodiment are described.

As described with FIG. 10 , in the detection system 10 according to thefourth embodiment, the direction of the line of sight is estimated fromthe positional relation between the eyelids and the eye. In the presentembodiment in particular, since the positional relation between theeyelids and the iris and the pupil can be appropriately calculated fromfeature points corresponding to the eyelids and feature figurescorresponding to the iris and the pupil, the direction of the line ofsight can be appropriately calculated.

Fifth Embodiment

A detection system 10 according to a fifth embodiment is described withreference to FIGS. 11 to 13 . The fifth embodiment is different fromeach of the above-described embodiments only in part of configurationand operation, and, for example, a hardware configuration may be similarto that of the first embodiment (see FIG. 1 ). Accordingly, in thefollowing, a description of part overlapping with the embodimentsdescribed already is omitted as appropriate.

(Functional Configuration)

First, a functional configuration of the detection system 10 accordingto the fifth embodiment is described with reference to FIG. 11 . FIG. 11is a block diagram showing the functional configuration of the detectionsystem according to the fifth embodiment. Note that in FIG. 11 ,elements similar to the constitutional elements shown in FIGS. 2, 5, and8 are denoted by the same reference numbers as in FIGS. 2, 5, and 8 .

As shown in FIG. 11 , the detection system 10 according to the fifthembodiment includes, as processing blocks for implementing functions ofthe detection system 10, or as physical processing circuitry, an imageacquisition unit 110, a detection unit 120, an angle-of-rotationestimation unit 150, and an image rotation unit 160. In other words, thedetection system 10 according to the fifth embodiment further includesthe angle-of-rotation estimation unit 150 and the image rotation unit160, in addition to the constitutional elements in the first embodiment(see FIG. 2 ). Note that the angle-of-rotation estimation unit 150 andthe image rotation unit 160 can be implemented by, for example, theabove-described processor 11 (see FIG. 1 ).

The angle-of-rotation estimation unit 150 can estimate an angle ofrotation (that is, a slope) of an image acquired by the imageacquisition unit 110, based on feature points detected by the detectionunit 120. For example, when feature points of eyelids are detected asillustrated in the second embodiment and the like (see FIG. 7 and thelike), the angle-of-rotation estimation unit 150 estimates the angle ofrotation of the image from a slope of the detected eyelids. Note thatthe angle-of-rotation estimation unit 150 may be configured to estimatethe angle of rotation of the image by taking a feature figure intoconsideration, in addition to the feature points detected by thedetection unit 120. For example, when feature points of eyelids and afeature figure of an iris or a pupil are detected as illustrated in thesecond embodiment and the like (see FIG. 7 and the like), theangle-of-rotation estimation unit 150 may estimate the angle of rotationof the image from a positional relation between the detected eyelids andthe detected iris and pupil.

The image rotation unit 160 is configured to be able to rotate the imageacquired by the image acquisition unit 110, based on the angle ofrotation estimated by the angle-of-rotation estimation unit 150. Inother words, the image rotation unit 160 is configured to be able toexecute image slope correction, based on the estimated angle ofrotation. The image rotation unit 160 may include a function of storingthe rotated image as the image after corrected.

(Flow of Operation)

Next, a flow of operation of the detection system 10 according to thefifth embodiment is described with reference to FIG. 12 . FIG. 12 is aflowchart showing the flow of the operation of the detection systemaccording to the fifth embodiment. Note that in FIG. 12 , processessimilar to the processes shown in FIGS. 3, 6, and 9 are denoted by thesame reference numbers as in FIGS. 3, 6, and 9 .

As shown in FIG. 12 , when the detection system 10 according to thefifth embodiment operates, first, the image acquisition unit 110acquires an image (step S101). Thereafter, the detection unit 120detects a feature figure corresponding to the first part from the imageacquired by the image acquisition unit 110 (step S102). The detectionunit 120 further detects feature points corresponding to the second partfrom the image acquired by the image acquisition unit 110 (step S103).

Subsequently, the angle-of-rotation estimation unit 150 estimates anangle of rotation of the image, based on the detected feature points(step S401). The image rotation unit 160 rotates the image, based on theestimated angle of rotation (step S402). The image rotation unit 160, inparticular, rotates the image around, as an axis of rotation, the centerof an approximate circle detected as the feature figure.

(Specific Example of Operation)

Next, a specific example of operation (that is, an example of operationof rotating an image) performed by the detection system 10 according tothe fifth embodiment is described with reference to FIG. 13 . FIG. 13shows the specific example of the operation performed by the detectionsystem according to the fifth embodiment.

As shown in FIG. 13 , in the detection system 10 according to the fifthembodiment, the angle of rotation of an image is estimated from thefeature points of the eyelids. In the example shown in the drawing, itcan be seen that the image slopes down to the left (counterclockwise).Note that a numerical value of the angle of rotation can be calculated,for example, by comparing preset positions of the feature points at anormal time and positions of the feature points currently detected.However, for a scheme of estimating an angle of rotation based onfeature points, an existing technique can be adopted as appropriate.

Subsequently, the image rotation unit 160 rotates the image by theestimated angle of rotation. In the example shown in the drawing, theimage rotation unit 160 rotates the image to the right (clockwise). Theimage rotation unit 160, in particular, rotates the image around, as anaxis of rotation, the center of the circle corresponding to the iris orthe pupil detected as a feature figure. Note that when a plurality offeature figures are detected (for example, when irises or pupils of botheyes are detected), the image rotation unit 160 may rotate the image byusing the center of any one of the feature figures for an axis ofrotation.

Technical Effects

Next, technical effects achieved by the detection system 10 according tothe fifth embodiment are described.

As described with FIGS. 11 to 13 , in the detection system 10 accordingto the fifth embodiment, the angle of rotation of an image is estimatedbased on detected feature points, and the image is rotated around, as anaxis of rotation, the center of a feature figure. With the configurationthus made, even when an image acquired by the image acquisition unit 110slopes, such a slope can be appropriately corrected. Note that the imageafter rotated can also be used for, for example, the iris recognitiondescribed in the third embodiment, and the line-of-sight estimationdescribed in the fourth embodiment. In such a case, since the slope hasbeen corrected by rotating the image, the iris recognition and theline-of-sight estimation can be executed with higher accuracy.

Sixth Embodiment

A detection system 10 according to a sixth embodiment is described withreference to FIGS. 14 to 20 . Note that the sixth embodiment isdifferent from each of the above-described embodiments only in part ofconfiguration and operation, and, for example, a hardware configurationmay be similar to that of the first embodiment (see FIG. 1 ).Accordingly, in the following, a description of part overlapping withthe embodiments described already is omitted as appropriate.

(Functional Configuration)

First, a functional configuration of the detection system 10 accordingto the sixth embodiment is described with reference to FIG. 14 . FIG. 14is a block diagram showing the functional configuration of the detectionsystem according to the sixth embodiment. Note that in FIG. 14 ,elements similar to the constituent elements shown in FIGS. 2, 5, 8, and11 are denoted by the same reference numbers as in FIGS. 2, 5, 8, and 11.

As shown in FIG. 14 , the detection system 10 according to the sixthembodiment includes, as processing blocks for implementing functions ofthe detection system 10, or as physical processing circuitry, an imageacquisition unit 110, a detection unit 120, and a display unit 170. Inother words, the detection system 10 according to the sixth embodimentfurther includes the display unit 170, in addition to the constitutionalelements in the first embodiment (see FIG. 2 ).

The display unit 170 is configured as, for example, a monitor includinga display. The display unit 170 may be configured as part of the outputdevice 16 shown in FIG. 1 . The display unit 170 is configured to beable to display information related to a feature figure and featurepoints detected by the detection unit 120. The display unit 170 may beconfigured to be able to change display forms, for example, according toan operation made by a system user or the like.

(Flow of Operation)

Next, a flow of operation of the detection system 10 according to thesixth embodiment is described with reference to FIG. 15 . FIG. 15 is aflowchart showing the flow of the operation of the detection systemaccording to the sixth embodiment. Note that in FIG. 15 , processessimilar to the processes shown in FIGS. 3, 6, 9, and 12 are denoted bythe same reference numbers as in FIGS. 3, 6, 9, and 12 .

As shown in FIG. 15 , when the detection system 10 according to thesixth embodiment operates, first, the image acquisition unit 110acquires an image (step S101). Thereafter, the detection unit 120detects a feature figure corresponding to the first part from the imageacquired by the image acquisition unit 110 (step S102). The detectionunit 120 further detects feature points corresponding to the second partfrom the image acquired by the image acquisition unit 110 (step S103).

Subsequently, the display unit 170 displays information related to thedetected feature figure and feature points (step S501). Not only theinformation directly related to the feature figure and the featurepoints, the display unit 170 may also display information that can beestimated from the feature figure and the feature points, or the like.

Display Examples

Next, examples of display rendered by the detection system 10 accordingto the sixth embodiment are described with reference to FIGS. 16 to 20 .Note that each display example described below may be used incombination as appropriate.

First Display Example

A first display example is described with reference to FIG. 16 . FIG. 16is a diagram (version 1) showing an example of display of feature pointsand feature figures on the display unit.

As shown in FIG. 16 , the display unit 170 may display an image overwhich the feature figures and the feature points are drawn. In such acase, the display unit 170 may be configured to display only the featurefigures or the feature points. The display unit 170 may be configured toswitch between presence and absence of the drawn feature figures andfeature points, for example, according to an operation made by the user.When such an operation is configured to be made by the user, the displayunit 170 may be configured to display an operation button (that is, abutton for switching display) under the image or the like.

The display unit 170 may further display information indicatingpositions of the feature figures and the feature points (for example,coordinates of the feature points, formulas of the feature figures, andthe like), in addition to the feature figures and the feature points.Moreover, the display unit 170 may display the feature figures and thefeature points that are colored or demarcated such that ranges ofregions that can be specified by the feature figures and the featurepoints (in the example in the drawing, an eyelid region, an iris region,and a pupil region) can be identified.

Second Display Example

A second display example is described with reference to FIG. 17 . FIG.17 is a diagram (version 2) showing an example of display of the featurepoints and the feature figures on the display unit.

As shown in FIG. 17 , the display unit 170 may display an original image(that is, an input image) and a result of detection (that is, an imageobtained by drawing the feature figures and the feature points over theinput image) that are juxtaposed to each other. In such a case, thedisplay unit 170 may be configured to display only any one of theoriginal image and the result of detection, for example, according to anoperation made by the user. Respective display forms for the originalimage and the result of detection may be configured to be able to bechanged individually.

Third Display Example

A third display example is described with reference to FIG. 18 . FIG. 18is a diagram (version 3) showing an example of display of the featurepoints and the feature figures on the display unit. Note that thedisplay example in FIG. 18 is intended for the second embodiment (thatis, the configuration including the iris recognition unit 130).

As shown in FIG. 18 , the display unit 170 may display a registeredimage for iris recognition and a currently picked-up image (that is, animage obtained by drawing the feature figures and the feature pointsover an input image) that are juxtaposed to each other. In such a case,the display unit 170 may be configured to display only any one of theregistered image and the picked-up image, for example, according to anoperation made by the user. Respective display forms for the registeredimage and the picked-up image may be configured to be able to be changedindividually.

Fourth Display Example

A fourth display example is described with reference to FIG. 19 . FIG.19 is a diagram (version 4) showing an example of display of the featurepoints and the feature figures on the display unit. Note that thedisplay example in FIG. 19 is intended for the third and fourthembodiments (that is, the configuration including the line-of-sightestimation unit 140).

As shown in FIG. 19 , the display unit 170 may be configured to displaya result of estimating a direction of a line of sight, in addition to animage over which the feature figures and the feature points are drawn.Specifically, the display unit 170 may display an arrow indicating thedirection of the line of sight, as shown in the drawing. In such a case,a configuration may be made such that the more the line of sightdeviates from the front, the longer or larger the arrow is made ondisplay.

Fifth Display Example

A fifth display example is described with reference to FIG. 20 . FIG. 20is a diagram (version 5) showing an example of display of the featurepoints and the feature figures on the display unit. Note that thedisplay example in FIG. 20 is intended for the fifth embodiment (thatis, the configuration including the angle-of-rotation estimation unit150 and the image rotation unit 160).

As shown in FIG. 20 , the display unit 170 may display an image beforerotated (that is, an image before a slope thereof is corrected) and theimage after rotated (that is, the image after the slope thereof iscorrected) that are juxtaposed to each other. In such a case, thedisplay unit 170 may be configured to display only any one of the imagebefore rotated and the image after rotated, for example, according to anoperation made by the user. Respective display forms for the imagebefore rotated and the image after rotated may be configured to be ableto be changed individually.

Technical Effects

Next, technical effects achieved by the detection system 10 according tothe sixth embodiment are described.

As described with FIGS. 14 to 20 , in the detection system 10 accordingto the sixth embodiment, information related to a detected featurefigure and detected feature points is displayed. Accordingly, a resultof detecting the feature figure and the feature points and results ofvarious processes using the feature figure and the feature points can bepresented to the user in an easily understandable manner.

Seventh Embodiment

A detection system 10 according to a seventh embodiment is describedwith reference to FIGS. 21 and 22 . Note that the seventh embodiment isdifferent from each of the above-described embodiments only in part ofconfiguration and operation, and, for example, a hardware configurationmay be similar to that of the first embodiment (see FIG. 1 ).Accordingly, in the following, a description of part overlapping withthe embodiments described already is omitted as appropriate.

(Functional Configuration)

First, a functional configuration of the detection system 10 accordingto the seventh embodiment is described with reference to FIG. 21 . FIG.21 is a block diagram showing the functional configuration of thedetection system according to the seventh embodiment. Note that in FIG.21 , elements similar to the constitutional elements shown in FIGS. 2,5, 8, 11, and 14 are denoted by the same reference numbers as in FIGS.2, 5, 8, 11, and 14 .

As shown in FIG. 21 , the detection system 10 according to the seventhembodiment includes, as processing blocks for implementing functions ofthe detection system 10, or as physical processing circuitry, an imageacquisition unit 110, a detection unit 120, and a learning unit 180. Inother words, the detection system 10 according to the seventh embodimentfurther includes the learning unit 180, in addition to theconstitutional elements in the first embodiment (see FIG. 2 ).

The learning unit 180 is configured to be able to learn a model (forexample, a neural network model) for detecting a feature figure andfeature points. When learning by the learning unit 180 is executed, theimage acquisition unit 110 acquires an image that is training data. Thelearning unit 180 executes learning by using a feature figure andfeature points detected by the detection unit 120 from the trainingdata. In other words, the learning unit 180 executes learning by usingthe detected feature figure and feature points for a composite target.More specifically, the learning unit 180 executes learning by comparinga feature figure and feature points detected by the detection unit 120,to correct data on the feature figure and the feature points inputted asthe training data. The learning unit 180 may be configured to have partof learning processes be executed outside of the system (for example, beexecuted by an external server, cloud computing, or the like).

Note that the detection system 10 according to the seventh embodimentmay include a function of augmenting the inputted training data. Forexample, the image acquisition unit 110 may perform data augmentation,such as luminance change, vertical/horizontal shift, scaling, androtation.

(Flow of Operation)

Next, a flow of operation of the detection system 10 according to theseventh embodiment is described with reference to FIG. 22 . FIG. 22 is aflowchart showing the flow of the operation of the detection systemaccording to the seventh embodiment. Note that in FIG. 22 , processessimilar to the processes shown in FIGS. 3, 6, 9, 12, and 15 are denotedby the same reference numbers as in FIGS. 3, 6, 9, 12, and 15 .

As shown in FIG. 22 , when the detection system 10 according to theseventh embodiment operates, first, the image acquisition unit 110acquires an image (step S101). Thereafter, the detection unit 120detects a feature figure corresponding to the first part from the imageacquired by the image acquisition unit 110 (step S102). The detectionunit 120 further detects feature points corresponding to the second partfrom the image acquired by the image acquisition unit 110 (step S103).

Subsequently, the learning unit 180 calculates an error function fromthe detected feature figure and feature points (step S601).Specifically, the learning unit 180 calculates distances between vectorsindicating the detected feature figure and feature points, and vectorsindicating the feature figure and the feature points in the trainingdata (that is, the correct data), and thereby calculates errors betweenthe detected feature figure and feature points and the correct data. Fora scheme of calculating the errors, for example, L1 norm or L2 norm canbe used, but another scheme may be used.

Subsequently, the learning unit 180 performs error backpropagation basedon the errors, and calculates a gradient of a parameter of the detectionmodel (step S602). Thereafter, the learning unit 180 updates (optimizes)the parameter of the detection model, based on the calculated gradient(step S603). For a scheme of optimization, for example, SDG (StochasticGradient Descent), Adam, or the like can be used, but optimization maybe performed by using another scheme. When optimizing the parameter, thelearning unit 180 may perform regularization such as weight decay. Whenthe detection model is a neural network, a layer may be included thatperforms regularization such as dropout or batch norm.

Note that the above-described series of learning processes (that is,steps S601 to S603) is only an example, and the learning may be executedby using another scheme as long as a feature figure and feature pointscan be used for a composite target.

Lastly, the learning unit 180 determines whether or not the learning isfinished (step S604). The learning unit 180 determines whether or notthe learning is finished, for example, based on whether or not apredetermined number of loops of the processes up to here are executed.When it is determined that the learning is finished (step S604: YES),the series of processes is terminated. When it is determined that thelearning is not finished (step S604: NO), the processes are repeatedfrom step S101.

Technical Effects

Next, technical effects achieved by the detection system 10 according tothe seventh embodiment are described.

As described with FIGS. 21 and 22 , in the detection system 10 accordingto the seventh embodiment, the learning is executed by using a featurefigure and feature points for a composite target. Accordingly, the modelthat detects a feature figure and feature points can be optimized, andmore appropriate detection can be realized.

Modification

Here, a modification of the above-described seventh embodiment isdescribed. A configuration and operation of the modification areapproximately similar to those of the seventh embodiment describedalready. Accordingly, in the following, part different from the seventhembodiment is described in detail, and a description of other part isomitted as appropriate.

In a detection system 10 according to the modification, the learningunit 180 executes the learning processes by using information related adistribution of a relative positional relation between a feature figureand feature points. For example, the learning unit 180 learns the modelfor detecting a feature figure and feature points by using distributionsof positions of an iris detected as the feature figure, and positions ofeyelids detected as the feature points.

Here, if the learning using the relative positional relation between thefeature figure and the feature points is not performed, there is apossibility that a part that is not the iris is detected as the iris, ora part that is not the eyelids is detected as the eyelids, because eachof the iris and the eyelids is detected independently of each other(that is, the relative positional relation is not taken intoconsideration).

However, according to the detection system 10 in the modification, sincethe learning is executed by taking into consideration the relativepositional relation between the feature figure and the feature points,the model for detecting a feature figure and feature points can be moreappropriately optimized.

<Supplementary Notes>

The above-described embodiments can also be further described as, butare not limited to, the following supplements.

(Supplementary Note 1)

A detection system described in Supplementary Note 1 is a detectionsystem including: an acquisition unit configured to acquire an imageincluding a living body; and a detection unit configured to detect, fromthe image, a feature figure corresponding to an appropriately circularfirst part on the living body, and feature points corresponding to asecond part around the first part on the living body.

(Supplementary Note 2)

A detection system described in Supplementary Note 2 is the detectionsystem described in Supplementary Note 1, further comprising an irisrecognition unit configured to execute an iris recognition process onthe living body, based on the feature figure corresponding to at leastone of an iris and a pupil that are the first parts, and on the featurepoints corresponding to eyelids that are the second part.

(Supplementary Note 3)

A detection system described in Supplementary Note 3 is the detectionsystem described in Supplementary Note 1 or 2, further comprising aline-of-sight estimation unit configured to execute a line-of-sightestimation process of estimating a line of sight of the living body,based on the feature figure corresponding to at least one of an iris anda pupil that are the first parts, and on the feature pointscorresponding to eyelids that are the second part.

(Supplementary Note 4)

A detection system described in Supplementary Note 4 is the detectionsystem described in Supplementary Note 3, wherein the line-of-sightestimation unit is configured to estimate the line of sight of theliving body, based on a relative positional relation between the featurefigure corresponding to at least one of the iris and the pupil and thefeature points corresponding to the eyelids.

(Supplementary Note 5)

A detection system described in Supplementary Note 5 is the detectionsystem described in any one of Supplementary Notes 1 to 4, furthercomprising: an angle-of-rotation estimation unit configured to estimatean angle of rotation of the image by using the feature pointscorresponding to eyelids that are the second part; and an image rotationunit configured to rotate the image by the angle of rotation around, asan axis of rotation, a center of the feature figure corresponding to atleast one of an iris and a pupil that are the first parts.

(Supplementary Note 6)

A detection system described in Supplementary Note 6 is the detectionsystem described in any one of Supplementary Notes 1 to 5, furthercomprising a display unit configured to display the feature points andthe feature figure in a display form in which the feature points and thefeature figure can be individually identified.

(Supplementary Note 7)

A detection system described in Supplementary Note 7 is the detectionsystem described in any one of Supplementary Notes 1 to 6, furthercomprising a learning unit configured to execute a learning process forthe detection unit, by using the feature points and the feature figuredetected from the image that is training data.

(Supplementary Note 8)

A detection system described in Supplementary Note 8 is the detectionsystem described in Supplementary Note 6, wherein the learning unit isconfigured to execute the learning process by using information relatedto a relative positional relation between the feature figure and thefeature points.

(Supplementary Note 9)

A detection method described in Supplementary Note 9 is a detectionmethod including: acquiring an image including a living body; anddetecting, from the image, a feature figure corresponding to anappropriately circular first part on the living body, and feature pointscorresponding to a second part around the first part on the living body.

(Supplementary Note 10)

A computer program described in Supplementary Note 10 is a computerprogram that allows a computer to: acquire an image including a livingbody; and detect, from the image, a feature figure corresponding to anappropriately circular first part on the living body, and feature pointscorresponding to a second part around the first part on the living body.

(Supplementary Note 11)

A recording medium described in Supplementary Note 11 is a recordingmedium on which the computer program described in Supplementary Note 10is recorded.

Changes can be made to the present disclosure as appropriate within ascope that does not conflict with the gist or the principle of theinvention that can be read from the claims and the specification in itsentirety, and a detection system, a detection method, and a computerprogram with such changes are also incorporated within the technicalidea of the present disclosure.

DESCRIPTION OF REFERENCE CODES

-   -   10 Detection system    -   11 Processor    -   110 Image acquisition unit    -   120 Detection unit    -   130 Iris recognition unit    -   140 Line-of-sight estimation unit    -   150 Angle-of-rotation estimation unit    -   160 Image rotation unit    -   170 Display unit    -   180 Learning unit

What is claimed is:
 1. A detection system comprising: at least one memory configured to store instructions; and at least one processor configured to execute the instructions to: acquire an image including a living body; train a detection model composed of a neural network and adapted to detect, from the image, a feature figure corresponding to an iris of the living body, and feature points corresponding to an eyelid of the living body, such that the detection model takes into account a relational position between the feature figure and the features points to prevent a non-iris part of the image from being detected as the feature figure and to prevent a non-eyelid part of the image from being detected as the feature points, and such that the feature figure and the feature points are not detected independently of one another; detect the feature figure and the feature points within the image using the trained detection model; estimate an angle of rotation of the image using the feature points corresponding to the eyelid; and rotate the image by the estimated angle of rotation around a center of the feature figure corresponding to the iris as the axis of rotation.
 2. The detection system according to claim 1, wherein the at least one processor is configured to execute the instructions to further execute a line-of-sight estimation process of estimating a line of sight of the living body, based on the feature figure and the feature points.
 3. The detection system according to claim 2, wherein the line of sight of the living body is estimated based on the relational position between the feature figure and the feature points.
 4. The detection system according to claim 1, wherein the at least one processor is configured to execute the instructions to further display the feature points and the feature figure in a manner in which the feature points and the feature figure are individually identifiable.
 5. The detection system according to claim 1, wherein the detection model is further trained based on the relational position between the feature figure and the feature points.
 6. A detection method comprising: acquiring, by a processor, an image including a living body; training, by the processor, a detection model composed of a neural network and adapted to detect, from the image, a feature figure corresponding to an iris of the living body, and feature points corresponding to an eyelid of the living body, such that the detection model takes into account a relational position between the feature figure and the features points to prevent a non-iris part of the image from being detected as the feature figure and to prevent a non-eyelid part of the image from being detected as the feature points, and such that the feature figure and the feature points are not detected independently of one another; detecting, by the processor, the feature figure and the feature points within the image using the trained detection model; estimating, by the processor, an angle of rotation of the image using the feature points corresponding to the eyelid; and rotating, by the processor, the image by the estimated angle of rotation around a center of the feature figure corresponding to the iris as the axis of rotation.
 7. A non-transitory recording medium storing a computer program executable by a computer to: acquire an image including a living body; train a detection model composed of a neural network and adapted to detect, from the image, a feature figure corresponding to an iris of the living body, and feature points corresponding to an eyelid of the living body, such that the detection model takes into account a relational position between the feature figure and the features points to prevent a non-iris part of the image from being detected as the feature figure and to prevent a non-eyelid part of the image from being detected as the feature points, and such that the feature figure and the feature points are not detected independently of one another; detect the feature figure and the feature points within the image using the trained detection model; estimate an angle of rotation of the image using the feature points corresponding to the eyelid; and rotate the image by the estimated angle of rotation around a center of the feature figure corresponding to the iris as the axis of rotation. 